Dynamically configuring erasure code redundancy and distribution

ABSTRACT

Example apparatus and methods monitor conditions in a tiered storage system. The conditions monitored may include the availability of different numbers and types of devices including an erasure code based object storage system. The conditions monitored may also include the availability and type of devices available to the erasure code based object storage system. A redundancy policy for storing an item using the erasure code based object storage system may be determined based on the conditions. Erasure codes associated with the item may then be stored in the erasure code based object storage system as controlled, at least in part, by the redundancy policy. The redundancy policy for the erasure codes may be updated dynamically in response to changing conditions on the tiered storage system.

REFERENCE TO RELATED APPLICATION

This Application is a Continuation of U.S. application Ser. No.14/185,331 filed on Feb. 20, 2014, the contents of which is herebyincorporated by reference in its entirety.

BACKGROUND

Different approaches, devices, and collections of devices may be used toprotect files, information about files, or other electronic data. Forexample, a tiered archive system may use tape drives, disk drives, solidstate drives (SSD), and an object store to store a file, to storeinformation about a file, to store redundant copies of files, or tostore other electronic data.

To insure data protection, different approaches for storing redundantcopies of items have been employed. Erasure codes are one such approach.An erasure code is a forward error correction (FEC) code for the binaryerasure channel. The FEC facilitates transforming a message of k symbolsinto a longer message with n symbols such that the original message canbe recovered from a subset of the n symbols, k and n being integers,n>k. The original message may be, for example, a file. The fractionr=k/n is called the code rate, and the fraction k′/k, where k′ denotesthe number of symbols required for recovery, is called the receptionefficiency. Optimal erasure codes have the property that any k out ofthe n code word symbols are sufficient to recover the original message.Optimal codes may require extensive memory usage, CPU time, or otherresources when n is large.

Erasure codes are described in coding theory. Coding theory is the studyof the properties of codes and their fitness for a certain purpose(e.g., backing up files). Codes may be used for applications including,for example, data compression, cryptography, error-correction, andnetwork coding. Coding theory involves data compression, which may alsobe referred to as source coding, and error correction, which may also bereferred to as channel coding. Fountain codes are one type of erasurecode.

Fountain codes have the property that a potentially limitless sequenceof encoding symbols may be generated from a given set of source symbolsin a manner that supports ideally recovering the original source symbolsfrom any subset of the encoding symbols having a size equal to or largerthan the number of source symbols. A fountain code may be optimal if theoriginal k source symbols can be recovered from any k encoding symbols,k being an integer. Fountain codes may have efficient encoding anddecoding algorithms that support recovering the original k sourcesymbols from any k′ of the encoding symbols with high probability, wherek′ is just slightly larger than k. A rateless erasure code isdistinguished from an erasure code that exhibits a fixed code rate.

Object based storage systems may employ rateless erasure code technology(e.g., fountain codes) to provide a flexible level of data redundancy.The appropriate or even optimal level of data redundancy produced usinga rateless erasure code system may depend, for example, on the numberand type of devices available to the object based storage system. Theactual level of redundancy achieved using a rateless erasure code systemmay depend, for example, on the difference between the number ofreadable redundancy blocks (e.g., erasure codes) written by the systemand the number of redundancy blocks needed to reconstruct the originaldata. For example, if twenty redundancy blocks are written and onlyeleven redundancy blocks are needed to reconstruct the original datathat was protected by generating and writing the redundancy blocks, thenthe original data may be reconstructed even if nine of the redundancyblocks are damaged or otherwise unavailable.

Object based storage systems using rateless erasure code technology mayfacilitate storing erasure codes generated according to differentredundancy policies (e.g., 7/3, 20/9, 20/2). A redundancy policy may bereferred to as an N/M redundancy policy where N total erasure codes aregenerated and the message can be regenerated using any N-M of the Ntotal erasure codes, M and N being integers, M<N.

When an object storage is used in a tiered archive system, the overalldata redundancy achieved by the tiered archive system depends on thedistribution of data between devices and the redundancy policiesassociated with the object store. The distribution of data refers to thenumber and types of devices participating in storing data. Theredundancy policies may include an N/M policy. For example, if thetiered archive system includes a RAID-6 tier at one site, and if data ispresent in the RAID-6 tier, then an optimal overall data redundancymight be provided by implementing a 20/2 erasure code policy at anobject storage at a single other site. If the RAID-6 tier is released orbecomes otherwise unavailable, then the optimal overall data redundancymay be provided by an 18/8 erasure code policy spread across threesites. The “optimal” overall data redundancy may be defined by, forexample, a system administrator or data retention expert.

Since conditions in a tiered archive system can vary dynamically as, forexample, devices become available or unavailable, the optimal overalldata redundancy may also vary dynamically. However, conventional tieredarchive systems may use pre-defined configuration settings to staticallydetermine erasure code policies.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate various example systems, methods,and other example embodiments of various aspects of the invention. Itwill be appreciated that the illustrated element boundaries (e.g.,boxes, groups of boxes, or other shapes) in the figures represent oneexample of the boundaries. One of ordinary skill in the art willappreciate that in some examples one element may be designed as multipleelements or that multiple elements may be designed as one element. Insome examples, an element shown as an internal component of anotherelement may be implemented as an external component and vice versa.Furthermore, elements may not be drawn to scale.

FIG. 1 illustrates a tiered archive system with an object store.

FIG. 2 illustrates a tiered archive system with an object store thatuses erasure code technology.

FIG. 3 illustrates distributing a message according to initialconditions.

FIG. 4 illustrates re-distributing a message in response to changedconditions.

FIG. 5 illustrates an example distribution and redundancy logic.

FIG. 6 illustrates an example method associated with dynamicallyconfiguring erasure code redundancy.

FIG. 7 illustrates an example method associated with dynamicallyconfiguring erasure code redundancy.

FIG. 8 illustrates an example apparatus configured to dynamicallyconfigure erasure code redundancy.

FIG. 9 illustrates an example apparatus configured to dynamicallyconfigure erasure code redundancy.

FIG. 10 illustrates an example apparatus associated with dynamicallyconfiguring erasure code redundancy in a tiered archive system.

DETAILED DESCRIPTION

Example apparatus and methods account for dynamically changingconditions in a storage system that uses an erasure code based objectstore. Conditions may vary dynamically as, for example, devices becomeavailable or unavailable. Example apparatus and methods also account forthe fact that the optimal overall data redundancy may vary dynamicallyas a function of the conditions in the storage system. Whileconventional systems may use pre-defined configuration settings tostatically determine erasure code redundancy policies, example apparatusand methods may dynamically identify optimum overall redundancy based oncurrent conditions in a system. Example apparatus and methods mayidentify redundancy policies to be employed based on the current optimumoverall redundancy. The redundancy policies may identify N/M policiesthat control the number of erasure codes generated and the distributionof those erasure codes. In one embodiment, the redundancy policies maybe identified from user defined rules. In another embodiment, theredundancy policies may be identified from automated redundancy rules.In one embodiment, redundancy policies may be determined for an erasurecode based object store operating in a tiered archive system.

The redundancy level may control how many erasure codes above theminimum number of erasure codes needed to reconstruct an item arestored. Recall that using erasure codes, an N/M policy may be employedto store data, M and N being integers, M<N. The N/M policy refers to thefact that to reconstruct a message for which N erasure codes aregenerated, only N-M erasure codes may need to be accessed. While Nerasure codes may be generated, example apparatus and methods maycontrol how many of the N erasure codes are stored in an object storebased on the safety factor to be protected by the N erasure codes. TheN-M erasure codes may be stored on a single device or may be distributedbetween two or more devices.

Example apparatus and methods may provide or interact with a userinterface that facilitates identifying an N/M policy, a distributionrule, or a redundancy rule for a tiered archive system. A tiered archivesystem may provide hierarchical storage management where data isautomatically moved between storage media. Data may be moved based, forexample, on the cost of storing the data on different media, on howquickly the data needs to be retrieved, or based on other factors.Hierarchical storage management facilitates moving data between, forexample, high-speed storage devices like hard disk drive arrays andlow-speed storage devices like tape libraries. While it may be ideal tohave all data always available on the fastest possible storage device,this ideal may be too expensive to achieve. Hierarchical storagemanagement may migrate data files that are used most frequently tohigher speed access devices and may migrate data files that are usedleast frequently to lower speed access devices.

FIG. 1 illustrates a tiered archive system 100. A tiered archive systemmay include several devices arranged in different tiers to providedifferent levels of redundancy for an item to be stored. For example,archive system 100 includes a tape 110, a disk 120, a RAID system 130,an SSD 140, and an object store 150. Different tiered archive systemsmay include a greater or lesser number of a similar or different varietyof devices. The tiered archive system 100 may store a message 160.Message 160 may be, for example, a file, a record, an object, a table,or other electronic data. Multiple copies of message 160 may be storedin archive system 100. For example, a copy of message 160 may be storedon tape 110, two copies may be stored on disk 120, a copy may bedistributed throughout RAID system 130, and a copy may be stored onobject store 150. Not all devices in an archive system 100 may beemployed to store an item.

FIG. 2 illustrates another embodiment of a tiered archive system 100.This embodiment includes an object store 170 that stores items usingrateless erasure codes (e.g., fountain codes). The rateless erasurecodes for message 160 may be produced by an erasure code generator 180and then selectively provided to the erasure code object store 170. Theerasure code object store 170 may include one or more devices (e.g.,disks, SSD). How many erasure codes are generated by erasure codegenerator 180 may be controlled by an N/M redundancy policy. In oneembodiment, the N/M redundancy policy may be determined based on thestatus (e.g., availability, capacity, reliability) of devices in thearchive system 100. In one embodiment, the N/M redundancy policy may bedetermined based on the status (e.g., availability, capacity,reliability) of devices in the erasure code object store 170. In oneembodiment, the N/M redundancy policy may be determined based on thestatus (e.g., availability, capacity, reliability) of devices in thearchive system 100 and the status of devices in the erasure code objectstore 170.

FIG. 3 illustrates distributing a message 250 according to initialconditions in a tiered archive system 200. The tiered archive system 200may include a RAID system 210, a non-erasure code based object store220, and an erasure code based object store 230. Different archivesystems may include different combinations of devices. Redundancycontroller 260 may receive message 250 and be tasked with storingmessage 250 according to a desired redundancy level. In one embodiment,the desired redundancy level may be an optimal redundancy level. Thedefinition of “optimal” at any given time may depend on the status ofthe archive system 200. For example, if RAID 210 is functioning and hassufficient capacity to store message 250, then the optimal redundancylevel may reflect the availability and capacity of RAID 210. Similarly,if the object store 220 and the erasure code object store 230 areavailable, then the optimal redundancy level may reflect theavailability and capacity of those devices or systems. Redundancycontroller 260 may receive message 250, evaluate the conditions in thearchive system 200, and determine how to distribute the message 250between the devices in the archive system 200. Redundancy controller 260may also determine an N/M redundancy policy for erasure codes generatedfor the message 250. The erasure codes may be generated by the erasurecode generator 235 and provided to the erasure code object store 230.

Conditions may change in a tiered archive system. FIG. 4 illustratesre-distributing message 250 in response to changed conditions in thetiered archive system 200. For example, RAID 210 may have becomeunavailable. Thus, redundancy logic 260 may determine a new N/Mredundancy policy.

FIG. 5 illustrates an example redundancy logic 510 gathering informationfrom a device 520, an object store 530, and an erasure code based objectstore 540. Device 520, object store 530, and erasure code based objectstore 540 may be part of a tiered archive system. Rather than determinean N/M redundancy policy based on static or pre-configured assumptions,redundancy logic 510 may make decisions based on data received fromdevice 520, object store 530, erasure code object store 540, or otherdevices or systems associated with a tiered archive system.

Redundancy logic 510 may receive a message 500 and be tasked with makingsure that message 500 is stored in a manner that achieves a minimal oroptimal redundancy. The redundancy may be determined by the number andtype of devices to which the message 500 is written. The redundancy mayalso be determined by the number of erasure codes that are generated andwritten for the message. The redundancy may also be determined by thenumber of devices to which the erasure codes are distributed. In oneembodiment, after discovering information concerning devices in thetiered archive system, redundancy logic 510 may determine replicationpatterns, redundancy policies, or redundancy patterns based onuser-defined rules 512. The user-defined rules 512 may define thedesired patterns or policies for different possible configurations andconditions of a tiered archive system. In one embodiment, afterdiscovering information concerning the tiered archive system, redundancylogic 510 may determine replication patterns, redundancy policies, orredundancy patterns based on automated rules 514. The automated rules514 may seek to maximize or even optimize redundancy for message 500.

Some portions of the detailed descriptions herein are presented in termsof algorithms and symbolic representations of operations on data bitswithin a memory. These algorithmic descriptions and representations areused by those skilled in the art to convey the substance of their workto others. An algorithm, here and generally, is conceived to be asequence of operations that produce a result. The operations may includephysical manipulations of physical quantities. Usually, though notnecessarily, the physical quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated. The physical manipulations create aconcrete, tangible, useful, real-world result.

It has proven convenient at times, principally for reasons of commonusage, to refer to these signals as bits, values, elements, symbols,characters, terms, or numbers. It should be borne in mind, however, thatthese and similar terms are to be associated with the appropriatephysical quantities and are merely convenient labels applied to thesequantities. Unless specifically stated otherwise, it is to beappreciated that throughout the description, terms including processing,computing, and determining refer to actions and processes of a computersystem, logic, processor, or similar electronic device that manipulatesand transforms data represented as physical (electronic) quantities.

Example methods may be better appreciated with reference to flowdiagrams. For purposes of simplicity of explanation, the illustratedmethodologies are shown and described as a series of blocks. However, itis to be appreciated that the methodologies are not limited by the orderof the blocks, as some blocks can occur in different orders orconcurrently with other blocks from that shown and described. Moreover,less than all the illustrated blocks may be required to implement anexample methodology. Blocks may be combined or separated into multiplecomponents. Furthermore, additional or alternative methodologies canemploy additional, not illustrated blocks.

FIG. 6 illustrates a method 600 associated with dynamically configuringerasure code redundancy. In one embodiment, the erasure code redundancymay be computed for an erasure code based object store in a tieredarchive system. Method 600 may include, at 610, accessing availabilitydata concerning a storage device. The storage device may be located in atiered archive system. The tiered archive system may include an erasurecode based object store. In one embodiment, accessing the availabilitydata includes querying the storage device. In another embodiment,accessing the availability data may include receiving the data from thestorage device without querying the device. Thus, the availability datamay be acquired, among other ways, by using a pull model or by operationof a push model from the tiered storage system. Accessing theavailability data may include, for example, reading a computer memory,receiving data in a function call, reading data from a register, orreading data from another computer-readable storage medium.

Method 600 may also include, at 620, identifying an N/M redundancypolicy for storing a message using the erasure code based object store.The N/M redundancy policy may be based, at least in part, on theavailability data. Recall that an N/M redundancy policy identifies aminimum number N-M of erasure codes needed to reconstruct the messageand a maximum number N of erasure codes to be generated for the message,M being less than N, M and N being integers.

In one embodiment, the N/M redundancy policy may be a function of a userdefined rule. For example, a user may define the desired number oferasure codes to be produced and the desired distribution of the erasurecodes for different anticipated conditions in the tiered storage system.All possible conditions may not be able to be anticipated. Or, a usermay choose not to provide a comprehensive set of rules, or even anyrules. Thus, in one embodiment, identifying the N/M redundancy policymay be based, at least in part, on an automated rule. The automated rulemay identify a minimum number of erasure codes to be present on aminimum number of different devices associated with the erasure codebased object store.

Method 600 may also include, at 660, selectively causing an erasure codeassociated with the item to be stored on the erasure code based objectstore. The number of erasure codes written at 660 may be controlled bythe N/M policy.

At 680, a determination is made concerning whether updated data isavailable. If updated data is not available, then method 600 may simplywait until updated data becomes available. But if updated data isavailable, then method 600 may return to earlier actions to identify anupdated N/M redundancy policy. The updated N/M redundancy policy may bebased on the updated data. In addition to selecting an updated N/Mredundancy policy, method 600 may identify an updated object storedevice to use for storing an erasure code associated with the message.Which object store device is selected as the updated object store devicemay depend on the updated N/M redundancy policy.

In different embodiments, the determination at 680 may be made after aperiodic check of the availability data, upon detecting a changedcondition in the tiered archive system, or after receiving a controlsignal from the tiered archive system.

FIG. 7 illustrates another embodiment of method 600. This embodiment ofmethod 600 also includes, at 650, generating the erasure code. Theerasure code may be generated according to the N/M policy. Generatingthe erasure code may include controlling, for example, a circuit orprocess to input the message and output the erasure codes.

This embodiment of method 600 also includes, at 670, selectively causingthe message to be stored on the non-object store device(s). Causing themessage to be stored on the non-object store may include, for example,writing the message to a shared memory, writing the message to a memory,writing the message to a network interface, or providing the message toanother computer-readable storage medium.

The following includes definitions of selected terms employed herein.The definitions include various examples and/or forms of components thatfall within the scope of a term and that may be used for implementation.The examples are not intended to be limiting. Both singular and pluralforms of terms may be within the definitions.

References to “one embodiment”, “an embodiment”, “one example”, “anexample”, and other similar terms, indicate that the embodiment(s) orexample(s) so described may include a particular feature, structure,characteristic, property, element, or limitation, but that not everyembodiment or example necessarily includes that particular feature,structure, characteristic, property, element or limitation. Furthermore,repeated use of the phrase “in one embodiment” does not necessarilyrefer to the same embodiment, though it may.

ASIC: application specific integrated circuit.

CD: compact disk.

CD-R: CD recordable.

CD-RW: CD rewriteable.

DVD: digital versatile disk and/or digital video disk.

HTTP: hypertext transfer protocol.

LAN: local area network.

RAM: random access memory.

DRAM: dynamic RAM.

SRAM: synchronous RAM.

RAID: redundant array of independent disks.

ROM: read only memory.

PROM: programmable ROM.

SSD: solid state drive

SAN: storage area network.

USB: universal serial bus.

WAN: wide area network.

“Computer component”, as used herein, refers to a computer-relatedentity (e.g., hardware, firmware, software in execution, combinationsthereof). Computer components may include, for example, a processrunning on a processor, a processor, an object, an executable, a threadof execution, and a computer. A computer component(s) may reside withina process and/or thread. A computer component may be localized on onecomputer and/or may be distributed between multiple computers.

“Computer-readable storage medium”, as used herein, refers to anon-transitory medium that stores instructions and/or data. Acomputer-readable medium may take forms, including, but not limited to,non-volatile media, and volatile media. Non-volatile media may include,for example, optical disks, magnetic disks, and other disks. Volatilemedia may include, for example, semiconductor memories, dynamic memory,and other memories. Common forms of a computer-readable medium mayinclude, but are not limited to, a floppy disk, a flexible disk, a harddisk, a magnetic tape, other magnetic medium, an ASIC, a CD, otheroptical medium, a RAM, a ROM, a memory chip or card, a memory stick, andother media from which a computer, a processor or other electronicdevice can read.

“Data store”, as used herein, refers to a physical and/or logical entitythat can store data. A data store may be, for example, a database, atable, a file, a data structure (e.g. a list, a queue, a heap, a tree) amemory, a register, or other repository. In different examples, a datastore may reside in one logical and/or physical entity and/or may bedistributed between two or more logical and/or physical entities.

“Logic”, as used herein, includes but is not limited to hardware,firmware, software in execution on a machine, and/or combinations ofeach to perform a function(s) or an action(s), and/or to cause afunction or action from another logic, method, and/or system. Logic mayinclude, for example, a software controlled microprocessor, a discretelogic (e.g., ASIC), an analog circuit, a digital circuit, a programmedlogic device, or a memory device containing instructions. Logic mayinclude one or more gates, combinations of gates, or other circuitcomponents. Where multiple logical logics are described, it may bepossible to incorporate the multiple logical logics into one physicallogic. Similarly, where a single logical logic is described, it may bepossible to distribute that single logical logic between multiplephysical logics.

An “operable connection”, or a connection by which entities are“operably connected”, is one in which signals, physical communications,or logical communications may be sent or received. An operableconnection may include a physical interface, an electrical interface, ora data interface. An operable connection may include differingcombinations of interfaces or connections sufficient to allow operablecontrol. For example, two entities can be operably connected tocommunicate signals to each other directly or through one or moreintermediate entities (e.g., processor, operating system, logic,software). Logical or physical communication channels can be used tocreate an operable connection.

“Signal”, as used herein, includes but is not limited to, electricalsignals, optical signals, analog signals, digital signals, data,computer instructions, processor instructions, messages, a bit, or a bitstream, that can be received, transmitted and/or detected.

“Software”, as used herein, includes but is not limited to, one or moreexecutable instructions that cause a computer, processor, or otherelectronic device to perform functions, actions and/or behave in adesired manner. “Software” does not refer to stored instructions beingclaimed as stored instructions per se (e.g., a program listing). Theinstructions may be embodied in various forms including routines,algorithms, modules, methods, threads, or programs including separateapplications or code from dynamically linked libraries.

“User”, as used herein, includes but is not limited to one or morepersons, software, logics, applications, computers or other devices, orcombinations of these.

FIG. 8 illustrates an apparatus 800 that includes a processor 810, amemory 820, and a set 830 of logics that is connected to the processor810 and memory 820 by an interface 840. In one embodiment, the apparatus800 may be an object storage system. In one embodiment, the apparatus800 may be operably connected to or in data communication with an objectstorage system. Recall that an object storage system performsobject-based storage using a storage architecture that manages data asobjects instead of, for example, as files. “Object”, as used herein,refers to the usage of object in computer science. From one point ofview, an object may be considered to be a location in a physical memoryhaving a value and referenced by an identifier.

The set 830 of logics may include a first logic 832 that produces amonitor data concerning storage devices associated with a tiered storagesystem. The tiered storage system may include multiple storage devices(e.g., tape, disk, RAID, object stores). Additionally, the tieredstorage system may include an erasure code based object storage system.Producing the monitor data may include, for example, writing electronicvalues to a computer memory (e.g., RAM, DRAM), writing values to aregister, writing values to a disk, or storing other computer-readabledata on a computer-readable storage medium.

In one embodiment, the first logic 832 produces the monitor data byacquiring status information for a device or devices in the tieredstorage system. In one embodiment, the status information may beacquired periodically. In another embodiment, the status information maybe acquired in response to receiving a request to store an item in thetiered storage system. In another embodiment, the status information maybe sent from the tiered storage system in response to detecting a changein the status (e.g., availability, capacity, reliability) of a device inthe tiered storage system.

The apparatus 800 may also include a second logic 834 that determines aredundancy policy for storing erasure codes for an item. The redundancypolicy may depend on the monitor data. The erasure codes will be storedusing the erasure code based object storage system.

In one embodiment, the second logic 834 determines the redundancy policybased, at least in part, on the number or type of non-object storagedevices in the tiered storage system. For example, when a large numberof devices are participating in storing redundant copies of the item,the second logic 834 may produce a first N/M redundancy policy that doesnot rely heavily on an erasure code based object store while when asmaller number of devices are participating in storing redundant copiesthe second logic 834 may produce a second N/M policy that relies moreheavily on the erasure code based object store. In one embodiment, thesecond logic 834 may determine the redundancy policy based, at least inpart, on the number or type of devices in the erasure code based objectstorage system. For example, when an erasure code based object storagesystem only has a single disk, then a first N/M policy may be employedwhile when multiple disks are available, a different N/M policy may beemployed.

The apparatus 800 may also include a third logic 836 that stores theerasure code or codes associated with the item in the erasure code basedobject storage system. The third logic 836 may store the erasure code orcodes as controlled, at least in part, by the redundancy policy.

In one embodiment, the first logic 832 controls the second logic 834 tore-determine the redundancy policy upon detecting a change in themonitor data. For example, when a device becomes (un)available, theredundancy policy may be updated to account for the change in status ofthe device. The first logic 832 may also control the third logic 836 tore-store erasure codes associated with the item upon detecting thechange in the monitor data.

FIG. 9 illustrates another embodiment of apparatus 800. This embodimentincludes a fourth logic 838. The fourth logic 838 may produce theerasure codes for the item. In one embodiment, the fourth logic 838 iscontrolled, at least in part, by the redundancy policy. Producing theerasure codes may include, for example, receiving a message, accessingthe redundancy policy, and writing erasure codes to a computer-readablestorage medium.

FIG. 10 illustrates an example computing device in which example systemsand methods described herein, and equivalents, may operate. The examplecomputing device may be a computer 1000 that includes a processor 1002,a memory 1004, and input/output ports 1010 operably connected by a bus1008. In one example, the computer 1000 may include a redundancy logic1030 that controls the distribution of a message within a tiered archivesystem that produces a redundancy policy for generating and writingerasure codes for the message, and that controls the distribution oferasure codes between devices available to an object store. In differentexamples, the logic 1030 may be implemented in hardware, software,firmware, and/or combinations thereof. While the logic 1030 isillustrated as a hardware component attached to the bus 1008, it is tobe appreciated that in one example, the logic 1030 could be implementedin the processor 1002.

In one embodiment, logic 1030 may provide means (e.g., hardware,software, firmware, circuit) for identifying the number and types ofdevices available to participate in storing a message in a tieredarchive system. The tiered archive system may include an erasure codebased object store in which erasure codes associated with the messagemay be stored. The erasure code based object store may have one or moredevices (e.g., disks) on which the erasure codes may be stored.

Logic 1030 may also provide means (e.g., hardware, software, firmware,circuit) for determining a number of erasure codes to be written for themessage in the erasure code based object store. The number of erasurecodes to be written may depend on the number and types of devicesavailable to store the message and on the number and types of devicesavailable to the erasure code based object store.

Logic 1030 may also provide means (e.g., hardware, software, firmware,circuit) for determining a number of devices in the erasure code basedobject store to which the number of erasure codes are to be distributed.In one embodiment, the number of erasure codes and the number of devicesmay be manipulated to optimize a redundancy measure for the message. Inone embodiment, the number of erasure codes and the number of devicesmay be manipulated to improve a redundancy measure for the message. Inone embodiment, the number of erasure codes and the number of devicesmay be manipulated to provide a minimum acceptable redundancy in theshortest possible amount of time.

The means associated with logic 1030 may be implemented, for example, asan ASIC that implements the functionality of apparatus described herein.The means may also be implemented as computer executable instructionsthat implement the functionality of methods described herein and thatare presented to computer 1000 as data 1016 that are temporarily storedin memory 1004 and then executed by processor 1002.

Generally describing an example configuration of the computer 1000, theprocessor 1002 may be a variety of various processors including dualmicroprocessor and other multi-processor architectures. A memory 1004may include volatile memory and/or non-volatile memory. Non-volatilememory may include, for example, ROM, PROM, and other memory. Volatilememory may include, for example, RAM, SRAM, DRAM, and other memory.

A disk 1006 may be operably connected to the computer 1000 via, forexample, an input/output interface (e.g., card, device) 1018 and aninput/output port 1010. The disk 1006 may be, for example, a magneticdisk drive, a solid state disk drive, a floppy disk drive, a tape drive,a Zip drive, a flash memory card, a memory stick, or other device.Furthermore, the disk 1006 may be a CD-ROM drive, a CD-R drive, a CD-RWdrive, a DVD ROM drive, a Blu-Ray drive, an HD-DVD drive, or otherdevice. The memory 1004 can store a process 1014 and/or a data 1016, forexample. The disk 1006 and/or the memory 1004 can store an operatingsystem that controls and allocates resources of the computer 1000.

The bus 1008 may be a single internal bus interconnect architectureand/or other bus or mesh architectures. While a single bus isillustrated, it is to be appreciated that the computer 1000 maycommunicate with various devices, logics, and peripherals using otherbusses (e.g., PCIE, 1394, USB, Ethernet). The bus 1008 can be typesincluding, for example, a memory bus, a memory controller, a peripheralbus, an external bus, a crossbar switch, and/or a local bus.

The computer 1000 may interact with input/output devices via the i/ointerfaces 1018 and the input/output ports 1010. Input/output devicesmay be, for example, a keyboard, a microphone, a pointing and selectiondevice, cameras, video cards, displays, the disk 1006, the networkdevices 1020, and other devices. The input/output ports 1010 mayinclude, for example, serial ports, parallel ports, and USB ports.

The computer 1000 can operate in a network environment and thus may beconnected to the network devices 1020 via the i/o interfaces 1018,and/or the i/o ports 1010. Through the network devices 1020, thecomputer 1000 may interact with a network. Through the network, thecomputer 1000 may be logically connected to remote computers. Networkswith which the computer 1000 may interact include, but are not limitedto, a LAN, a WAN, and other networks.

While example systems, methods, and other embodiments have beenillustrated by describing examples, and while the examples have beendescribed in considerable detail, it is not the intention of theapplicants to restrict or in any way limit the scope of the appendedclaims to such detail. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the systems, methods, and other embodiments described herein.Therefore, the invention is not limited to the specific details, therepresentative apparatus, and illustrative examples shown and described.Thus, this application is intended to embrace alterations,modifications, and variations that fall within the scope of the appendedclaims.

To the extent that the term “includes” or “including” is employed in thedetailed description or the claims, it is intended to be inclusive in amanner similar to the term “comprising” as that term is interpreted whenemployed as a transitional word in a claim.

To the extent that the term “or” is employed in the detailed descriptionor claims (e.g., A or B) it is intended to mean “A or B or both”. Whenthe applicants intend to indicate “only A or B but not both” then theterm “only A or B but not both” will be employed. Thus, use of the term“or” herein is the inclusive, and not the exclusive use. See, Bryan A.Garner, A Dictionary of Modern Legal Usage 624 (2d. Ed. 1995).

What is claimed is:
 1. An apparatus, comprising: a processor; a memory;a set of circuits; and an interface configured to connect the processor,the memory, and the set of circuits; the set of circuits comprising: afirst circuit configured to produce a monitor data concerning storagedevices associated with a tiered storage system, where the tieredstorage system includes two or more storage devices arranged indifferent tiers, and where the two or more storage devices include anerasure code based object storage system or a non-object storage device,where the erasure code based object storage system or the non-objectstorage device includes a tape drive, a disk drive, a RAID device, or asolid state device, where the apparatus is operably connected with thetiered storage system; a second circuit configured to determine, basedat least in part on the monitor data, a redundancy policy for storingone or more erasure codes associated with an item using the erasure codebased object storage system or the non-object storage device; and athird circuit configured to store the one or more erasure codesassociated with the item in the erasure code based object storage systemas controlled, at least in part, by the redundancy policy, where thefirst circuit is configured to produce the monitor data by acquiringstatus information for one or more devices in the tiered storage systemfrom the tiered storage system in response to receiving a request tostore an item in the tiered storage system, where the first circuit isconfigured to acquire the status information using a push model withoutquerying the one or more devices, where the first circuit is configuredto, upon detecting a change in the monitor data, control the secondcircuit to re-determine the redundancy policy, and where the firstcircuit is configured to, upon detecting the change in the monitor data,control the third circuit to re-store erasure codes associated with theitem.
 2. The apparatus of claim 1, where the different tiers providedifferent levels of redundancy at different costs.
 3. The apparatus ofclaim 1, where the second circuit is configured to determine theredundancy policy based, at least in part, on the number or type ofnon-object storage devices in the tiered storage system.
 4. Theapparatus of claim 1, where the second circuit is configured todetermine the redundancy policy based, at least in part, on the numberor type of devices in the erasure code based object storage system. 5.The apparatus of claim 3, where the second circuit is configured todetermine the redundancy policy based, at least in part, on the numberor type of devices in the erasure code based object storage system. 6.The apparatus of claim 1, where the monitor data includes anavailability of one or more devices in the tiered storage system, acapacity of the one or more devices in the tiered storage system, or areliability of the one or more devices in the tiered storage system. 7.The apparatus of claim 1, where the first circuit is configured toacquire the status information by querying the one or more devices. 8.The apparatus of claim 1, where the second circuit is configured to,upon determining that the tiered storage system includes a thresholdnumber of non-object storage devices: identify, based at least in parton the status information, a first N/M redundancy policy for storing theitem using the erasure code based object storage system and thenon-object storage device, where the first N/M redundancy policyidentifies a minimum number N-M of erasure codes needed to reconstructthe item and a maximum number N of erasure codes to be generated for theitem, M being less than N, M and N being integers; and control the thirdcircuit to selectively cause a first erasure code associated with theitem to be stored on the erasure code based object store, and toselectively cause a second erasure code associated with the item to bestored on the non-object storage device.
 9. The apparatus of claim 8,where the second circuit is configured to, upon determining that thetiered storage system includes less than the threshold number ofnon-object storage devices: identify, based at least in part on thestatus information, a second N/M redundancy policy for storing the itemusing the erasure code based object store, where the second N/Mredundancy policy identifies a minimum number N-M of erasure codesneeded to reconstruct the item and a maximum number N of erasure codesto be generated for the item, M being less than N, M and N beingintegers; and control the third circuit to selectively cause an erasurecode associated with the item to be stored on the erasure code basedobject store.
 10. The apparatus of claim 1, where the second circuit isconfigured to determine the redundancy policy based, at least in part,on a user defined rule.
 11. The apparatus of claim 1, where the secondcircuit is configured to determine the redundancy policy based, at leastin part, on an automated rule.
 12. The apparatus of claim 1, comprisinga fourth circuit configured to produce the erasure codes for the item,where the fourth circuit is configured to be controlled, at least inpart, by the redundancy policy.